1. Field of the Invention
This invention relates to a high-speed signal search method, device, and recording medium (which records this method) that searches a signal sequence for the location of a signal that is similar to a pre-recorded signal. This recording medium records a program for executing by computer this high-speed signal search method, and is computer readable. The present invention can be used, for example, for audio signal detection. That is, the present invention relates to signal detection technology which can detect and automatically record the time a specified commercial was broadcast in a broadcast audio signal, and start and stop video recording by detecting a specified theme song.
In addition, the present invention is related to technology that can automatically monitor the timing of applause, the timing of laughter, etc. in a broadcast, and search for a specified scene. Furthermore, the present invention can be applied to the detection of signals in general (video signals, etc.), not just audio signals.
This application is based patent applications filed in Japan (Japanese Patent Application, No. Hei 10-151723, Japanese Patent Application, No. Hei 10-244162, and Japanese Patent Application, No. Hei 11-49184), the contents of which are incorporated herein by reference.
2. Prior Art
A matched filter is a conventional technique of detecting in a signal sequence a region having a desired signal (referred to hereinbelow as a xe2x80x9creference signalxe2x80x9d). A matched filter is a technique wherein a correlation between the waveform of the area of the object input signal (referred to hereinbelow as a xe2x80x9cwindowxe2x80x9d) and the waveform of a reference signal is calculated as the window moves, and when the correlation value exceeds a fixed value, it is determined that the reference signal is within the window.
However, in this method, there is the problem that because it is necessary to calculate the correlation value while precisely moving the position of the window with respect to the entire area of the input signal, the amount of calculation becomes enormous, and the calculation speed becomes slow.
In contrast, instead of using the correlation value between the input signal waveform and a reference signal waveform, there is also the technique of calculating the feature quantities (that is, a numerical value or a set of numerical values indicating the feature of the signal waveform) of the input signal waveform in this window, and carrying out a comparison between this input signal waveform and the reference signal waveform by using, for example, the correlation value or Euclidean distance between this feature quantity and the feature quantity of a reference signal waveform calculated in advance.
However, in this method as well, there is the problem that it is necessary to calculate the feature quantities while precisely moving the position of the window with respect to the entire area of the input signal, and because it is necessary to compare feature quantities, the amount of calculation becomes enormous, and the speed of calculation is slow.
In consideration of the above-described problems, it is an object of the present invention to provide a high-speed signal search method, device, and recording medium for the same which can obtain a detection result equivalent to the case of precisely moving with respect to the entire area even if the window is not precisely moved with respect to the entire area of the input signal.
In addition, another object of the present invention is to provide a high-speed signal search method, device, and recording medium for the same which can detect a signal with higher precision even when it is difficult to discriminate the signal with average features or when the signal fluctuates due to noise, etc.
Furthermore, another object of the present invention is to provide a high-speed signal search method, device, and recording medium for the same which can detect a signal with less processing than conventionally even when the signal is detected based on a plurality of reference signals.
This invention provides a first step that generates a feature quantity sequence for a pre-recorded reference signal, a second step that sets the input signal window for the input signal that has been input, a third step that generates a feature quantity sequence for the input signal of this input signal window, a fourth step that calculates the input signal similarity value showing the degree of similarity between the feature quantity sequence generated in the first step and the feature quantity sequence generated in the third step, a fifth step that calculates the skip width showing the amount that the input signal window can be moved based on the input signal similarity value calculated in the fourth step, and a sixth step that determines the position of the input signal window based on the skip width calculated in the fifth step, sets the input signal window to this position, and further calculates the input signal similarity value for each position of the input signal window by repeating the third step to the sixth step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention provides a first step that generates a feature quantity sequence for a pre-recorded reference signal, a second step that produces a histogram of the feature quantity generated in the first step, a third step that sets the input signal window for the input signal that has been input, a fourth step that generates a feature quantity sequence for the input signal of this input signal window, a fifth step that produces a histogram of the feature quantity sequence generated in the fourth step, a sixth step that calculates the input signal similarity value showing the degree of similarity between the histogram produced in the second step and the histogram produced in the fifth step, a seventh step that calculates a skip width indicating the amount that the input signal window can be moved based on the input signal similarity value calculated in the sixth step, and an eight step that determines the position of the input signal window based on the skip width calculated in the seventh step, and sets the input signal window at that position, and further, calculates the input signal similarity value for each position of the input signal window by repeating the fourth step to the eighth step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention provides a first step that generates a feature quantity sequence for a pre-recorded reference signal, a second step that sets the reference signal window for the feature quantity sequence generated in the first step, a third step that partitions the reference signal window set in the second step into a plurality of partitioned reference signal windows, a fourth step that generates a feature quantity sequence for the input signal that has been input, a fifth step that sets the input signal window for the feature quantity sequence generated in the fourth step, a sixth step that partitions this input signal window into a plurality of partitioned input signal windows corresponding to the plurality of partitioned reference signal windows, a seventh step that calculates the input signal similarity value showing the degree of similarity between the feature quantity sequence of each reference signal partition window and the feature quantity sequence in the partitioned input signal windows corresponding to the relevant partitioned reference signal window, an eighth step that calculates a skip width indicating the amount that the input signal window can be moved based on the input signal similarity value calculated in the seventh step, and a ninth step which determines the position of the input signal window based on the skip width calculated in the eighth step and sets the input signal window at that position, and further, calculates an input signal similarity value for each position of the input signal window by repeating the sixth step to the ninth step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention provides a first step that generates a feature quantity sequence for a pre-recorded reference signal, a second step that sets a reference signal window for the feature quantity sequence generated in the first step, a third step that partitions the reference signal window set in the second step into a plurality of partitioned reference signal windows, a fourth step that generates a feature quantity sequence for the input signal that has been input, a fifth step that sets the input signal window for the feature quantity sequence generated in the fourth step, a sixth step that partitions this input signal window into a plurality if partitioned input signal windows corresponding to this plurality of partitioned reference signal windows, a seventh step that produces a histogram of the feature quantity sequence of each partitioned reference signal window, an eighth step the produces a histogram of a feature quantity sequence of each partitioned input signal window, a ninth step that calculates the input signal similarity value showing the degree of similarity between the histogram of each partitioned reference signal window and the histogram of the partitioned input signal window corresponding to the relevant partitioned reference signal window, a tenth step that calculates the skip width showing the amount that the input signal window can move based on the input signal similarity value calculated in the ninth step, and an eleventh step that determines the position of the input signal window and sets the input signal window at that position, and further, calculates an input signal similarity value for each position of the input signal window by repeating the sixth step to the eleventh step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention provides a first step that generates a feature quantity sequence for a plurality of pre-recorded reference signals, a second step that generates a feature quantity sequence for the input signal that has been input, a third step that sets the input signal window for the feature quantity sequence generated in the second step, a fourth step that calculates an inter-reference signal similarity value that shows the degree of similarity between the feature quantity sequence related to a former reference signal and the feature quantity sequence related to a later reference signal for two reference signals among the plurality of reference signals, a fifth step that calculates the input signal similarity value showing the degree of similarity between the feature quantity sequence generated in the first step and the feature quantity sequence in the said input signal window for each reference signal among said plurality of reference signals, a sixth step that calculates a skip width showing the amount that the input signal window can move based on the inter-reference signal similarity value calculated in the fourth step and the input signal similarity value calculated in the fifth step, and a seventh step that determines the position of the input signal window based on the skip width calculated in the sixth step and sets the input signal window at that position, and further, calculates an input signal similarity value for each position of the input signal window by repeating the fifth step to the seventh step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention provides a first step that generates a feature quantity sequence for a plurality of pre-recorded reference signals, a second step that produces histograms for the feature quantity sequences generated in the first step, a third step that generates a feature quantity sequence for an input signal that has been input, a fourth step that sets the input signal window for the feature quantity sequence generated in the third step, a fifth step that produces histograms for the feature quantity sequences of the input signal window, a sixth step that calculates an inter-reference signal similarity value showing the degree of similarity between the histogram related to a former reference signal and the histogram related to a later reference signal for two reference signals among the plurality of reference signals, a seventh step that calculates the input signal similarity value showing the degree of similarity between the histograms generated in the second step and the histograms generated in the fifth step for each reference signal among the plurality of reference signals, an eighth step that calculates a skip width showing the amount that the input signal window can move based on the inter-reference signal similarity value calculated in the sixth step and the input signal similarity value calculated in the seventh step, and a ninth step which determines the position of the input signal window based on the skip width calculated in the eighth step and sets the input signal window to this position, and further, calculates the input signal similarity value based on each position of the input signal window by repeating the fifth step to the ninth step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention provides a first step that generates a feature quantity sequence for a plurality of pre-recorded reference signals, a second step that sets the reference signal window for each feature quantity sequence generated in the first step, a third step that partitions the reference signal windows set in the second step into a plurality of partitioned reference signal windows, a fourth step that generates feature quantity sequence for the input signal that has been input, a fifth step that sets the input signal window for the feature quantity sequences generated in the fourth step, a sixth step that partitions this input signal window into a plurality of partitioned input signal windows corresponding to this plurality of partitioned reference signal windows, a seventh step calculates an inter-reference similarity value that is a similarity value showing the degree of similarity between the feature quantity sequence related to a former reference signal and the feature quantity sequence related to a later reference signal for two reference signals among the plurality of reference signals, and is a similarity value showing the degree of similarity between the feature quantity sequences of each partitioned reference signal window corresponding to each other between these two reference signals, an eighth that step calculates the input signal similarity value that shows the degree of similarity between the feature quantity sequence of each partitioned reference signal window and the feature quantity sequence of the partitioned input signal window corresponding to thus partitioned reference signal window for each reference signal among the plurality of reference signals, a ninth step that calculates a skip width indicating the amount that an input signal window can move based on the inter-reference signal similarity value calculated in the seventh step and the input signal similarity value calculated in the eighth step, and a tenth step which determines the position of the input signal window based on the skip width calculated in the ninth step and sets the input signal window at that position, and further, calculates an input signal similarity value for each position of the input signal window by repeating the sixth step to the tenth step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention provides a first step that generates a feature quantity sequence of a plurality of pre-recorded reference signals, a second step that sets a reference signal window for each feature quantity sequence generated in the first step, a third step that partitions the reference signal window set in the second step into a plurality of partitioned reference signal windows, a fourth step that generates a feature quantity sequence for the input signal that has been input, a fifth step that obtains an input signal window for the feature quantity sequence generated in the fourth step, a sixth step that partitions the input signal window into a plurality of partitioned input signal windows corresponding to the plurality of partitioned reference signal windows, a seventh step that produces a histogram of the feature quantity sequences of each of the partitioned input signal windows, an eighth step that produces a histogram of the feature quantity sequences of each of the partitioned input signal windows, a ninth step that calculates an inter-reference signal similarity value which is the similarity value showing the degree of similarity between the histogram related to a later reference signal and a histogram related to a former reference signal for two reference signals among this plurality of reference signals, and is the similarity value showing the degree of similarity between the histograms of each partitioned reference signal window corresponding to each other among the two inter-reference signals, a tenth step that calculates the input signal similarity value showing the degree of similarity between the histogram of each of the partitioned reference signal windows and the histogram of the partitioned input signal window corresponding to this partitioned reference signal window for each reference signal among the plurality of reference signals, an eleventh step that calculates a skip width indicating the amount that the input signal window can move based on the inter-reference signal similarity value calculated in the ninth step and the input signal similarity value calculated in the tenth step, and twelfth step that determines the position of the input signal window based on the skip width calculated in the eleventh step and sets the input signal window to that position, and further, calculates the input signal similarity value for each position of the input signal window by repeating the sixth step to the twelfth step, and determines whether or not the reference signal exists at the position that the input signal window presently shows in the input signal based on the result of comparing the input signal similarity value and the predetermined threshold value.
According to another feature, the present invention is a high-speed signal search device that carries out each of the above steps.
According to another feature, the present invention is an automatic video control system providing the high-speed signal search device, a video device, and a control means which controls the image movement of the video device based on the results of identification of this high-speed signal search device.
According to another feature, the present invention is a recording medium that records a program to execute each of the above steps.
According to the present invention, it is possible to obtain the detection results identical to the case when precisely moving over the whole area without precisely moving over the window for the entire area of the input signal. In addition, according to the present invention, it is possible to search for the time of the appearance of specified music or a commercial from among the signals over a long period of time of a broadcast, for example, and search for a signal including the specified signal from a signal database, for example.
In addition, according to the present invention, for a feature quantity sequence of a plurality of windows, it is possible to carry out a search taking into account the before-and-after relationship between these feature quantity sequences (in the temporal axis). Thereby, even when the signal fluctuates due to noise, etc and when the signal is difficult to discriminate with average features, it is possible to detect the signal with a higher precision. Moreover, as a setting configuration of the xe2x80x9cplurality of windowsxe2x80x9d, for example, partitioning the original window to make a plurality of windows that are set can be considered.
Furthermore, according to the present invention, it is possible to detect a signal by less processing that is conventional even which detecting a signal based on a plurality of reference signals.
Moreover, the program recorded on the recording medium of the present invention is read and executed by a computer, and thereby it is possible to detect a signal with less processing than is conventional, and an improvement in the calculation efficiency in the signal detection processing can be implemented.